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Abstract 

The notion of utility maximising entropy (it-entropy) of a probability den- 
sity, which was introduced and studied in [SZ04] , is extended in two di- 
rections. First, the relative it-entropy of two probability measures in ar- 
bitrary probability spaces is defined. Then, specialising to discrete prob- 
ability spaces, we also introduce the absolute it-entropy of a probability 
measure. Both notions are based on the idea, borrowed from mathemat- 
ical finance, of maximising the expected utility of the terminal wealth of 
an investor. Moreover, it-entropy is also relevant in thermodynamics, as it 
can replace the standard Boltzmann-Shannon entropy in the Second Law. 
If the utility function is logarithmic or isoelastic (a power function), then 
the well-known notions of the Boltzmann-Shannon and Renyi relative en- 
tropy are recovered. We establish the principal properties of relative and 
discrete u-entropy and discuss the links with several related approaches 
in the literature. 



1 Introduction 

The notion of utility maximising entropy (or u-entropy for brevity) of a density / 
of a probability distribution with respect to a given probability measure [i was 
introduced and studied by two of the present authors in SZ04 . 

The work in SZ04] was motivated, on the one hand, by problems in math- 
ematical finance concerned with a trader with a concave utility function u who 
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wants to maximise the expected utility E„(u(ui)) under the true market proba- 
bility measure v over all contingent claims w (where w is a non-negative random 
variable representing the final value of a contingent claim) whose initial value 
E^(w) under a pricing measure \i is equal to the initial wealth of the trader, taken 
to be 1 for simplicity, so that E M (i/;) = 1. A value c£l, called the certainty 
equivalent, can be assigned to each contingent claim w so that u(c) = E„ («(«;)). 
If v is absolutely continuous with respect to /i with density /, then the u-entropy 
H u (f) is defined as the highest possible value of the logarithm of the certainty 
equivalent c over all contingent claims w with initial value E Al (w) = 1. 

Expected utility maximisation problems in mathematical finance have been 
studied extensiv el y, for e xample, in |PK96j . |AIS98j . [REKOQj . jGROlj . [BF02] . 
[FB05J, [Gun05j, [GF06 . Some of the most general and elegant results, which 
have provided much inspiration for our work, belong to Kramkov and Schacher- 
mayer [KS99] . [KS03] . [SchOi] . [HKS05] . 

On the other hand, further motivation for u-entropy comes from thermody- 
namics and statistical mechanics. When u{x) = In a;, then the w-entropy H u (f) 
is equal to the classical Boltzmann-Gibbs entropy H{f) = E fJi (f\nf) (note the 
sign convention typical of mathematical literature; the opposite sign for entropy 
would normally be used in physics). The properties of the Boltzmann-Gibbs 
entropy H(f) and, in particular, its role in the Second Law of thermodynamics 
provided a fertile ground for generalisation to the case of u-entropy. As is very 
well known, a physical system in state / evolves towards equilibrium whenever 
H(f) tends to zero. An extension of this and other properties of entropy was 
achieved in |SZ04j by replacing the Boltzmann-Gibbs entropy H(f) with the 
u-entropy H u (f) for an arbitrary u from a broad class of utility functions. 

In the present paper the concept of it-entropy is extended further to include 
relative entropy of two arbitrary probability measures v and [i. It will be called 
the relative u-entropy and denoted by H u (y || We also introduce the u- 
entropy h u (p) of a probability measure p (rather than the relative entropy of 
one measure with respect to another or that of a density with respect to a given 
probability measure), but to do so need to specialise to the case of a discrete 
probability space, where p is a probability vector. 

We establish some properties of relative u-entropy and discrete u-entropy, 
and study their relationships with other similar approaches in the literature. 
In particular, we discuss a link with the recent work by Friedman, Huang and 
Sandow FHS07], and with a much older approach by Arimoto )Ari71] . which 
does not refer to utility maximisation explicitly but is based on a similar concept. 
These two approaches work in the discrete case only. Moreover, returning once 
again to general probability spaces, we also establish a connection of relative u- 
entropy with Frittelli's generalised distance between two probability measures, 
introduced in FriOOj to solve the dual convex problem in a utility maximisation 
framework for asset pricing in an incomplete market. 

It will prove convenient to adopt the convention oo • = — oo • = through- 
out this paper. 
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2 Utility maximising relative entropy 
2.1 Utility functions 

Definition 2.1 Let u : (0, oo)— >M. We call u a utility function whenever u 
satisfies the Inada conditions, that is, u is a strictly concave strictly increasing 
continuously differentiable function such that 

u' (0) :— lim u' (x) — oo, u' (oo) := lim u' (x) = . 

We shall also use the notation 

u (0) := lim u (x) , u (oo) :— lim u (x) . 

x^O x/^oo 

Proposition 2.2 The function I := (u') 1 : (0, oo) — > (0, oo) is strictly de- 
creasing and satisfies 

I (0) := lim / (x) = oo, / (oo) := lim I (ac) = . 

Definition 2.3 Let u : (0, oo) — > R be a utility function. The convex dual 

u* : (0, oo) — > R is defined by 

u * (y) = sup (u (x) - yx) (2.1) 

z>0 

for any y 6 (0, oo). We also put 

u*(0) := lim u*(x) — u(oo), u* (oo) := lim u* (x) = u(0). (2.2) 

If A > and s = 0, we put / (A/s) := and u* (A/s) := u (0), consistently 
with the adopted notation I (oo) = and u* (oo) = u (0). 

Example 2.4 Let 7 e (-00, 1). Define u : (0, 00) — > R by 

f i(P-l) fort e (0,oo) and 7 £ (-00,0) U (0,1) 
" ^ ~ \ 7 hit for t £ (0, 00) and 7 = 

We call u the isoelastic utility of order 7 if 7 ^ 0, and the logarithmic 
utility if 7 — 0. 

The following definition is due to Kramkov and Schachermayer KS99 . 

Definition 2.5 The asymptotic elasticity of a utility function u : (0, 00) — > 
R is defined by 

^ , , , am' (x) 

AE(tt) = hmsup — 1 . 

x/"oo M ( a; ) 

A utility function it is said to have reasonable asymptotic elasticity if 

AE(w) < 1. 

Under the assumption of reasonable asymptotic elasticity, duality theory for 
utility maximisation works in a similar manner as in the finite-dimensional case. 
See |Sch04j for equivalent formulations of this assumption and a discussion of 
its economic meaning. 
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2.2 Relative w-entropy and ^-entropy 
2.2.1 Definition 

Notation 2.6 Let (f2, E) be a measurable space. We denote by Mi(f2, E) the 
space of all probability measures on (f2,E). For any /i £ Mi(0, S) we denote 
by -D (/z) the set of all densities on the probability space (fi, E, n), that is, 



£(A0 := 



|u> £ L 1 (/i) : u> > and ^ u>d/z = 1 1 



By B (O, E) we denote the set of all bounded measurable real- valued functions on 
(fi, E). In the sequel we shall write simply Mi and B whenever the measurable 
space (fi, E) is unambiguous. For any fi £ Mi and / £ £KaO we shall write ffi 
to denote the measure in Mi with density / with respect to fi. 

Definition 2.7 Let u : (0, oo)^R be a utility function. Let (fi, E) be a mea- 
surable space and let fj, £ Mi (fi, E). We put 



N u (v || fi) := sup / zt (w) dv , 

where 

A (v, /i) := |u> G -D (/i) : u (w) _ £ i 1 (f)| . 

Here a; - = max(— x, 0) denotes the negative part of x G K. Note that 
u{w)dv G (—oo, oo] for each zu G A (v, it). We define 

H u {v || M ) ^lnn" 1 ^^ || M )) 

and call it the relative u-entropy (or relative utility maximising entropy) 

of v with respect to /i. 

Definition 2.8 (from |SZ04p Let u : (0, oo) -> 1 be a utility function and let 
fi G Mi. For any / G D(n) we put 



N u(f)-= sup / u{w)fdpi, 
weA{f) Jn 

where 

A (/) :={wED ( M ) : u (w)~ G L 1 (ffj,)} . 

Note that J Q u(w)f dfi G (— oo, oo] for each w £ A (/). We define 

H u (f) ^lnt^OM/)) 

and call it the w-entropy (utility maximising entropy) of /. 

The next proposition follows immediately from the definitions. 

Proposition 2.9 Let fi £ Mi and f £ D(/j,). Then 

N u (/) = N u (//, || M ) , 
H u (f) = H u (ffi\\fi) . 
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2.2.2 Properties 

Proposition 2.10 The following inequalities hold: 

u(l)<N u (v\\pi)<u(oc) , 
< H u (v || fj,) < oo . 

Proof Taking w = 1 G A (v, pi) , we obtain the lower bound. The upper bound 
follows immediately from the definition. ■ 

Proposition 2.11 Let pi, v\,v 2 G M\ and a G [0, 1]. Then 

N u (avx + (1 - a) v 2 \\ pi) < aN u (v x \\ pi) + (1 - a) N u (v 2 \\ pi) . 

Proof Put v := av\ + (1 — a)i>2. First observe that for w G D (pi) we 
have J n u~ (w) dv = a J n u~ (w) dv\ + (1 — a) J n u~ (w) dv 2 , and so A (v, pi) = 
A {v\ , pi) n A (v 2 , pi) . Hence 

N u (avi + (1 - a) v 2 || pi) 

— sup {j ' u (w) dv : w G A(v, pi) 

= sup <au (w) dv\ + (1 — a) / u (w) dv 2 : w G A(v, pi) 
I in Jn 

< a sup < / u (w) dv\ G A (v, pi) > + (1 — a) sup < / u (w) dv 2 : w G A(v, pi) 
[Jn ) [Jn 



<asup|y u (w) dv\ G A (v\, pi) j + (1 — a) sup <^J u(w) dv 2 : w G A(v 2 , pi) 
= aN u (u! || pi) + (1 - a) N u (v 2 \\ pi) , 
as desired. ■ 

Next we show that relative u-entropy can be reduced to the case when v <C pi- 
Theorem 2.12 Let pi,v G M x . Then 

N u (v || pi) = u ± (SI) u (oo) + v< (SI) N u f-^r- || pi 

\^ < (Sl) 

where v± + — v is the Lebesgue decomposition of v into the singular part v± 
and absolutely continuous part with respect to pi. 

Proof Let A G £ be such that pi (A) — and u± (A) = v± (SI). 

Step 1. If v <C pi, that is, v± (SI) — 0, then the assertion is trivial. Suppose 
that v _L pi, i.e., u± (SI) = 1. Then v (A) = 1 and w n := uIa + G -^(^m) 
for n G N, and J^u (w n ) dv = u(n). Hence and from Proposition 12 . 1 Ol we get 
N u (v || pi) = u (oo), as required. 
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Step 2. Now we assume that < v± (fi) < 1. Note that v = v± (fl) 
z/<g (fl) v ^q) , and from Proposition ^. Ill and from Step 1 we get 

N u (u || fi) < v ± (fl) u (oo) + v< (fl) JV U (^^y II /» 

Let now u> G .4 ^ v ^f(yi) » ^) • ^ >u * u '" := «1a + wIa 1 for ri G N. Clearly, 
w n G .4 (z/, //) and 

v± (fl) u (n) + v^ (fl) / u (w) d j— = I u (w n ) dv < N u (v || fi) . 

Jn v <, ("J 

Taking n — > oo completes the proof. ■ 

Corollary 2.13 In particular, ifu(oo) = oo and is is not absolutely continuous 
with respect to \x, or if v _L fi, then N u (v \\ fi) = u (oo) and H u (v || fi) = oo. 

Proposition 2.14 Let fx, v G M%. Then the following conditions are equivalent: 

(1) N u (v |[ fi) < u (oo) ; 

(2) N u (v || < oo ; 

(3) ff u (i/ || /x) < oo . 

In particular, all three conditions are satisfied for any utility function u such 
that u (oo) < oo. 

Proof The implications (1) =>■ (3) =>• (2) are obvious, as is (2) =>■ (1) when 
tt(oo) = oo. 

We shall prove 2) =>• 1) when u(oo) < oo. Put A n :— {w > n} for any 
n G N. Then HneN^™ = ^- Consequently, there exists an n G N such that 
v (A n ) =: 7 < 1. Hence 



u (w) dv — u (w) dv + u (w) dv 

n J A n J(A n ) n 

< u (oo) 7 + u (n) (1 — 7) 

for any w G A(v, fi). Thus N u (v || fi) < u (00) 7 + 71 (n) (1 — 7) < u (00), as 
required. ■ 

Proposition 2.15 The following conditions are equivalent: 

(1) H u (v || n) = ; 

(2) v = fi . 
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Proof (1) => (2). Let v = fj,. Take w E A{v,n). By Jensen's inequality 
J n u{w)dv < u (J" n wdv) = u (1). Hence, by Proposition ^. lOi N u (y || /i) = u(l), 
and so H u [y || fj) = 0. 

(2) =^> (1). Suppose that v ^ /x. Then there is an i 6 S such that ^ 
and < 1. We put 

1 - afJ,(A) 
w a := al A H l A n , 

) 

¥>(a):= / u(w a )dv = u(a)v{A) + u ( - — ^^-) i/(A") 



J 

for any a E (0, Clearly w a E A{v,n) and W\ = 1. Moreover, 

y>' (1) = u ' (1) ^Tfj^ ^ 0. Hence there exists an a E (0, l/fJ,(A)) such that 
it (w a ) dv = (p(a) > ip (1) = J n it (wi) dv ~ u (1). Thus ff u (V || /i) > 0. ■ 

Proposition 2.16 (linear transformation) Let u : (0, oo)— »R fee a utility 
function, let a > and /e£ fe S K. TTien u ~ au + b is a utility function, 
and for any i/,/i£ Mi 

N^{v || /i) =aiV„(^ || n) + b , 

Hu iy II aO = ff„ II aO • 

Proof This follows immediately from the definition. ■ 

Remark 2.17 It has recently been proved by Urbahski [Urb07] that in prob- 
ability spaces without atoms u — au + b is not only a sufficient condition, but 
in fact an equivalent condition for = H u . The equivalence can fail in a 
probability space with atoms. 

In |SZ01[ Theorem 20] we established a formula for u-entropy by convex du- 
ality methods. Namely, under the reasonable asymptotic elasticity assumption, 
if / S D (/i), then 

N u {f)= I u(I(A f /f))fd^ = [ u*(A f /f)fdn + Af , 

dfi 



H u (f) = lnu- 1 n u (I(A f /f))f 
where Af > is given implicitly as the unique solution of 

7(A///)d/i=l. 



Combined with Theorem 12.121 this makes it possible to evaluate the relative 
it-entropy H u (y || At) for any v,\iE M\. 



7 



Example 2.18 (logarithmic utility) Let u : (0, oo)^R be given by u (x) — 
lnx for x £ (0,oo). Then H u is equal to the Boltzmann-Shannon relative 
entropy 



Hi (y || fi) 



ffl da 

00 



otherwise 



for /i, v £ M\. 



Example 2.19 (isoelastic utility) Let u : (0, oo)-^R be given by u (x) — 
i (x 7 - 1) for 7 £ (-00, 0) U (0, 1) and x £ (0, 00). Then H u is equal to the 

Renyi relative entropy of order a = (1 — 7) 1 £ (0, 1) U (1, 00) given by 
^Ini^Y^ if 7 e (-oo,0), 



H a {y II m) 



for /1, v £ Mi. 



a— 1 Ji2 



dfi if 7 £ (0, 1) and v <C /i 
otherwise 



Remark 2.20 The Boltzmann-Shannon relative entropy was introduced in 
|KL51j under the name of directed divergence. It is also called the Kullback- 
Leibler divergence, relative information, conditional entropy, information gain 
or function of discrimination. The definition of the Renyi relative entropy (or 
divergence) of order a was proposed in |Ren61j . 



3 Discrete n-entropy 

Let fl = {uji, ■ ■ ■ ,Wfc} be a finite probability space equipped with the sigma- 
field E = 2 n of all subsets of il. The family of probability measures on 
(f2, S) will be denoted by Sk- For any p £ Sk we shall write pi = p(iOi) 
for i — l,...,k. Thus, we can identify Sk with the set of probability vec- 
tors |p £ R fc : J2i=i Pi = 1 an( l Pi > for i = 1, . . . , Our definition of the 
relative w-entropy covers also the discrete case. In this situation (though not 
necessarily in the general case) it is also possible to define the (non-relative) 
u-entropy as follows. 

Definition 3.1 Let u : (0, 00) — >R be a utility function, and let p £ Sk- Then 
we put 



n u (p) := sup V* u (wi)pi , 

and define the discrete it-entropy of p by 

h u (p) ■= - lnw -1 (n u (p)) . 
Remark 3.2 Note that h u depends only on the restriction of u to (0, 1] 
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Proposition 3.3 Let u : (0, oo)— ^R be a utility function. Let p £ Sk and let 

P(k) € Sk be the uniform probability vector, that is, (p(k))i = /or eac/i 
i = 1, . . . , k. Then 

H u (p || P(k)) = lnfc- h Uk (p) , 
where Uk : (0, oo)^R is ffce rescaled utility function 

Uk (x) := u {kx) (3.1) 

/or x £ (0, oo). 

Proof Since u^ 1 = ku^ 1 and 

k 

Nu(p\\P(k))= SU P X]"^^ 

»6 D (P{fc)) ,=i 

fc fe 

= sup ^u(kwi)p t = sup ^Ufc(wj)pi =n UJb (p) , 
wes k i=1 wes k i=1 

it follows that 

H u (p II P(k)) = h\u~ 1 {N u {p || p (fc) )) = ln^u^Vu^))] = lnfc - /i„ fc (p) . 

■ 

Using the above statement we can deduce many properties of discrete u- 
entropy from the respective properties of relative w-entropy. However, one can 
also prove them straightforwardly without assuming anything about the be- 
haviour of the function u outside the interval (0,1]. We could assume that 
u : (0, 1] — >M is a strictly concave strictly increasing continuously diffcrcntiable 
function such that lim^o u ' (x) = oo. In this case I := (u') _1 would be defined 
on the interval [it'(l),oo). The proofs of the following properties of discrete 
u-entropy are elementary. 

Proposition 3.4 Let p — {pi,.. . ,Pk) £ Sk- Then 

(1) < h u {p) < lnfc. 

(2) h u {p) = iff pi = 1 for some i = l,...,k. 

(3) h u {p) = lnfc iffp = P{k)- 

(4) h u {p) = ft u ((p w (i), . . . ,Pn(k))) f or ever y permutation n. 

(5) h u {p) = h u {{pi, . . . ,p k ,0)). 

(6) For a > and b £ R we have h au+ b = h u . 

The proof of the formula for u-entropy in the discrete case is also elementary 
and, by contrast to the general case, it does not require any further assumptions 
on u. 
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Proposition 3.5 Let p G Sk- Then: 

(1) There exists a unique A p > a (p) := v! (1) max p,- > suc/i £/ia£ 

j=l,...,k 

k 
i=l 

(2) TVie following formulae hold: 

k k 
nu (p)=Y, u ( J ( A pM)) Pi = XI u * iK/Pi)Pi + A p > ( 3 - 2 ) 

i=l i=l 
( k 

h u (p)=\vLU 1 22u(I (Ap/ pi)) pi 

\i=l 

Proof To prove (1) consider the function 4> p : [a (p) . oo) — > (0, oo) given 
by p (A) = X)i=i^( A /Pi) f° r A — Clearly, <j) p is continuous, strictly 

decreasing and satisfies 4> p (a(p)) — Yli=i I ( u> (l) max i=i, Pj/Pi) - ' 1 an d 
liniA^oo (A) = 0. As a result, there is a unique A p > a (p) such that <j) p (A p ) = 
1, as required. It follows from (1) that n u (p) > Yli=i u (I (A P /pi))pi- To prove 
the reverse inequality take w G Sk- Let i — 1, . . . , k. From the well-known 
formula u*(y) = u(I(y)) — yl(y) for the convex dual we get 

u (wi) - (Ap/pi) w l < u* ((Ap/pi)) =u(I ((Ap/pi))) - (Ap/pi) I ((Ap/ pi)) . 

(3-3) 

Multiplying (|3 . 3|) by p^, summing over i = 1, . . . , k, and adding A p , we obtain 
^u(wi)pi <^]ii* ((Ap/ p^) pi + A p = ^2u(I((A p /pi)))pi 

i—1 i—1 i—1 

Taking the supremum of the left-hand side over all such u>'s, we obtain the 
assertion. ■ 

Proposition 3.6 The function h u : Sk — > [0, lnfc] is continuous. 

Proof According to (|3.2p . it is enough to prove that Sk 3 P — > A p G [a (p) , oo) 
is continuous. Define F : {(p, A)} : p e S k ,A e [a (p) , oo)} — > R by F (p, A) = 
^* : =1 /(A/pj) — 1. Clearly, F is continuous, F(p,A p ) — for p G Sk, and 
[a (p) , oo) 3 A — > _F (p, A) G R is strictly decreasing for each p G Sk- Now, the 
assertion follows from the implicit function theorem for continuous functions. ■ 

Example 3.7 (logarithmic utility) Let u : (0, oo)— >R be given by u(x) — 
lax for x G (0, oo). Then the relative w-entropy H u (p || q) is equal to the 
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discrete Boltzmann-Shannon relative entropy (Kullback-Leibler di- 
vergence) 

{E Pi ln if V < Q ! 
oo otherwise 

for p, 5 6 Sfc, and the discrete it-entropy h u (p) is equal to the discrete 
Boltzmann-Shannon entropy 

k 

hi (p) = Pi In Pi 

for p E Sk- 

Example 3.8 (isoelastic utility) Let u : (0, oo)— >K be given by u (x) — 
- (x 7 — 1) for 7 G (—oo,0) U (0,1) and x G (0, oo). Then the discrete rela- 
tive u-entropy H u (p \\ Pqa) is equal to the discrete Renyi relative entropy 
(divergence) of order a = (1 — 7) 1 G (0, 1) U (1, 00) 

f rr'n E P?<t a if7 6(-oo,0), 

i=l,...,k 

h a {p\\q) = l ^T ln E P?9^ Q if 7£ (0,1) and p « q , 
»=i,...,fe 

00 otherwise 

for p, q G Sfe, and the discrete u-entropy /i„(p) is equal to the discrete Renyi 
entropy of order a 

i=l 

for p e S k - 

4 Relationships to other utility based concepts 
of entropy 

4.1 Priedman-Huang-Sandow £7 -entropy 

In [FHS07] (see also [FHS05]) the authors defined two quantities, which they 
called the [/-entropy and [/-relative entropy, noting their similarity to the u- 
entropy defined (in a much more general setting) in [SZ04j. In fact the [/-entropy 
and [/-relative entropy of Friedman, Huang and Sandow FHS07J, FHS05] can 
be reduced by a simple transformation to the relative u-entropy discussed in 
the present paper, and so to the u-entropy defined in [SZ04]. As a result, the 
properties of [/-entropy and [/-relative entropy claimed in [FHS05] . [FHS07J 
turn out to be immediate corollaries of the results of [SZ04 , as shown below. 

In the notation of the present paper the definitions in [FHS07] take the 
following form. 
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Definition 4.1 (Definition 5 from [FHS07) Let u : (0, oo)^M be a utility 
function and let p,q £ Sk- If p <C q, then the Friedman-Huang-Sandow U- 
relative entropy of p with respect to q is defined by 

D u (p || (?) := sup V u ( — ) - u (1) . 

Remark 4.2 By contrast to |FHS07| . it is not assumed here that u(l) = 0. 
To compensate, we subtract on the right-hand side of the formula defining 
D u (p || q)- The same applies to the formula defining H u (p) below. Moreover, 
instead of w £ Sk it is only assumed in |FHS07] that 53i=i w i ~ I> but presum- 
ably there is also a silent assumption that Wi j qi belongs to the domain of u for 
each i. In our case this means that, additionally, w > 0, so that w G Sk- The 
definitions and results easily extend to utility functions defined on an interval 
(a, b) other than (0, oo). If p is not absolutely continuous with respect to q, then 
D u {p II l) is undefined. 

Remark 4.3 The relative entropy defined in [FHS07J coincides with the deci- 
sion maker's optimal expected utility introduced in [JNW07, p. 13]. 

Definition 4.4 (Definition 6 from [FHS07] ) Let u : (0, oo)^K be a utility 
function and let p £ Sk- Then the Friedman-Huang-Sandow [/-entropy of p 
is defined by 

H u (p) = u (ft) - u (1) - D u (p || p {k )) ■ 

Proposition 4.5 Let u : (0, cx))^M be a utility function. Then the following 
properties hold: 

(1) For any p,q £ Sk such that pC? 

D u (p\\q)=u(e H ^W)-u(l) . 

(2) For any p £ Sk 

H u (p)=u k (l)-Uk(e- h ^) , 
where Uk is the rescaled utility function defined by §3.1$ . 

Proof (1) If p <C q, then wq € Sk is equivalent to w £ A(p, q). Hence 

k 

D u (p || q) = sup u (w i /q l )p l - u (1) 
wes k f=1 

= SUp ^ tt (Wj) Pi-U (1) = iV M (p || g) - tt (1) ■ 

weA(p,q) i=1 

The claim follows since N u (p || q) = u (e H »(p\\<i)) . 



12 



(2) This follows immediately from (1) and Proposition [ 

H u (p) = u(fc) -«(1) - D u (p || P(k) ) =u{k)-u ( e H "(pllP( fc ))) 

■ 

The [/-relative entropy D u (p \\ q) and the (7-entropy H u (p) of Friedman 
Huang and Sandow [FHS07] are therefore related to the w-entropy of |SZ04j by 

D u {p || q) = u(eMfV) -u(l) 

for each p,q 6 S k such that pCg, and by 



H u (p) = u (k) 



for each p € Sk- Because of this, the following results in [FHS07 are immediate 
consequences of the corresponding earlier results in |SZ04| : 

[SZ04] [FHS07j 

Theorem 20 =>• Lemma 1 

Propositions 8 and 10 =>• Corollary l.(i) 

Proposition 13 => Corollary l.(ii), (iv) 

Theorem 23 =>• Corollaries 3, 6 and 7 

Remark 4.6 The results of |SZ04j are valid in a much more general situation 
of arbitrary probability spaces, which requires the asymptotic elasticity assump- 
tion 

, ,. xu'(x) 
AEftt) := hmsup — < 1 

x/-oo U{X) 

to hold. In the discrete case this assumption is unnecessary and all the argu- 
ments in SZ04 work without it. 



4.2 Arimoto entropy 

A similar construction of entropy was first proposed by Arimoto [Ari71j without 
any explicit reference to the notion of utility. 

Definition 4.7 (Arimoto |Ari71j . see also |Tan07j ) For a non-negative function 
/ : (0, 1] — > M continuously differentiable on (0, 1] and such that /(l) = 
Arimoto's entropy is defined by 

k 

Hf(p):= inf 

for p e Sk- 



VS 



This was further generalized in |SS74| and also interpreted in [M PV96L 
Example 6] in terms of prior Bayes risk, where / plays the role of an individual 
uncertainty function. Arimoto's entropy is related to the entropy H u defined 
in FHS07J (see Definition 14.41 above) and to h u (Definition 13. ip as follows. 

Proposition 4.8 Let u : (0, oo)^]R be a utility function such that u (1) = 0, 
and let p € Sk- Then 

Ht(p)=H Ul/k (p) = -u(e- h ^). 
Proof This follows immediately from the definitions and Proposition ^. 51 (2): 

k k 



H-u (p) = inf V [-u (wi)} p l = - sup V u (w l )p t 
= -n u ( P ) = -u(e- h ^) = H Ul/k (p) . 



Example 4.9 (logarithmic utility) Let u be the logarithmic utility. For 

p,q e S k and p < q we have D u (p \\ q) = h 1 (p \\ q) and H u (p) = H^ u (p) = 

K (p). 

Example 4.10 (isoelastic utility) Let u be the isoelastic utility of order 7 £ 
(-00, 0) U (0, 1) and let a = (1 - 7)~\ 

1. For p,q G Sk with p <C q 

is proportional to the Sharma-Mittal relative entropy of order a 
and degree 2 — 1/a ; 

2. For pe S k 

H,M = ^^- a ((gpf) 5 -1) 

= i^ff^ , 

where is called the Arimoto entropy of kind 1/a. 

Remark 4.11 The Sharma-Mittal relative entropy was introduced in |SM75j . 
In [JNW07] the Sharma-Mittal relative entropy of order a and degree 2 — 1/a 
is called the pseudospherical divergence of order a. The Arimoto entropy 
was introduced by Arimoto [Ari71j and further elaborated in |BvdL80j . see 
also |Tan07j . 
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4.3 Prittelli generalised distance 

This notion of generalised distance in the set of probability measures was in- 
troduced in FriOQ] as a tool for solving the convex dual problem to that of 
computing the value of a financial security consistent with the no-arbitrage 
principle in an incomplete market model in a utility maximisation framework. 

Definition 4.12 (Definition 9 and formula (9) from [FriOQ] ) Let /x, v E M 1 be 
such that fi <C v. Then put 



A u (n, v) := sup [A+[ u* ( 

A>o \ Jn \ dv 



dp 



where u* is the convex dual to the utility function u given by (|2.1[) and 
and define the Frittelli generalised distance by 

8 u (n, v) := u^ 1 (A M (/x, v)) - 1 . 

Remark 4.13 Note the different (but equivalent) conventions as compared 
to [FriOO] . The differences lie in using the supremum rather than infimum 
coupled with different signs of certain expressions in the definitions of u* and 
A u (/x, v). The quantities A u (n, v) and S u (/j,,v) are denoted by A(/x, v\ 1) and 
S(n,v;l) in prOO] . 

Proposition 4.14 For any /i, v G M 1 such that /i <C v 



A u (n, v) = u(oo)^(0) + p^(Q)A u [ fj, 



i/<(n) 



where vj_ + v<^ — v is the Lebesgue decomposition of v into the singular part i/± 
and absolutely continuous part v<^ with respect to fi. 

Proof Because /x <C v, it follows that = a.s. with respect to v\_ and 
i/<g(ri) > 0. We can assume that v±(Q) > 0, since otherwise the assertion is 
obvious. Put i/<r = v Jr,\ ■ As a result, 



A„(/x, v) = sup ( A + / u* (A 
a>o V 7n V 



= sup ( A + / u* I A 3^ ) cZi/_l + [ u* \A-j^ I 



A>0 \ Jo V Jf2 



g?z^ 7 /o V (if 



= p x (Q)u* (0) + sup ( — ^— + / U * ^ ~7oT ~ff^~ ) "'" 

= ^_i_(Q)u (oo) + i^<(n) sup ( A + / u* f A-^- ) di/< 



A>0 \ ./<> V ^< 

2/x(fi)it(oo) + ^«(fi)A tl (/i, i^) 
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Proposition 4.15 Ifu has reasonable asymptotic elasticity, then for any fi,v G 
M such that fi <C v 



A u (fi, v) = N u (v || fi) . 
6 u (u,fi)=e H -^ -1 . 

Proof First we shall prove the proposition in the case when fi, v € M 1 are 
equivalent measures. Let f — j^- By Proposition 12.91 above and by Lemma 17 
and Theorem 20.4 in |SZ04j . we then have 



N u (y || fi) = N u (/) = sup A + / u* (A//) fdfi 
a>o V Jn 



sup ^A + J u* f A^J dvj = A u (fi, v) 



Now for any fi, v s A/ 1 such that <C ^ we take the Lebesgue decomposition 
f = z/j_ + ^< into the singular part i>±_ and absolutely continuous part z/<g 
with respect to /i. Then and f<g are equivalent measures. It follows by 
Propositions 12.121 and 14.141 that 

N u || fi) = u x (fi) u (oo) + ^« (fi) Ar u f || /i 



v± (fi) u (oo) + (fi) A u ( fi, V< J I = A„ (fi, v) 



v« («) 

The equality 5 U (y, fi) = e Hu ( v " fi ) — 1 now follows immediately from the defini- 
tions of 5 U (y, fi) and H u (y || fi). m 



5 Concluding remarks 

The notion of u-entropy of a probability density, based on the concept of ex- 
pected utility maximisation in finance, was first introduced in [SZ04] and linked 
with the Second Law of thermodynamics. In this paper the definition of u- 
entropy has been extended, on the one hand, to the case of relative M-entropy of 
one probability measure with respect to another, and, on the other hand, in the 
discrete case, to absolute it-entropy of a probability measure. Having established 
the basic properties of these notions, we have studied the relationships with 
other entropy-like quantities of a similar kind that can be found in the litera- 
ture. In particular, although all these approaches yield the Boltzmann-Shannon 
entropy when the logarithmic utility is used, it is only the relative u-entropy 
introduced in Definition 12.71 that is consistent with the Renyi entropy for isoe- 
lastic utility functions. The relationships between the various approaches are 
summarized in the diagram below. In this context, relative u-entropy emerges as 
the general unifying quantity among the various approaches related to expected 
utility maximisation. 
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general 
case 



discrete 
case 



H u (f) 



ThmEDJ 



PropEJ 



H u {y || [i] 



H u (p || q) 



Prop 



Prop 



h u (p) 



Prop |4~T5l 



Propg^l) 



PropgU;2) 



S u (v, fj,) 

D u (p || q) 
|FHS07j 

H u {p) 
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